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Abstract 

Varied sensory systems use noise in order to enhance detection of 
weak signals. It has been conjectured in the literature that this effect, 
known as stochastic resonance, may take place in central cognitive pro- 
cesses such as the memory retrieval of arithmetical multiplication. Wc 
show in a simplified model of cortical tissue, that complex arithmetical 
calculations can be carried out and are enhanced in the presence of a 
stochastic background. The performance is shown to be positively cor- 
related to the susceptibility of the network, defined as its sensitivity to 
a variation of the mean of its inputs. For nontrivial arithmetic tasks 
such as multiplication, stochastic resonance is an emergent property of 
the microcircuitry of the model network. 
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neural network 
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1 Introduction 

Noise is usually considered as having a corrupting effect on meaningful sig- 
nals. There is however one well known counter example to this widespread 
belief; the stochastic resonance (SR) phenomenon. In this case, addition of 
a random interference signal (noise) to a weak, subthreshold stimuli, may 
enhance its detection. Originally introduced in the framework of physics 
[UE1E1E1E1E1IZ1E] stochastic resonance is now known to take place in 
several sensory systems from the cricket cereal sensory system 0, to cray- 
fish mechanoreceptors jlU] , the somatosensory cortex of the cat jll] and the 
human visual cortex |12| ll.Sj in order to facilitate detection of weak signals 
(for a review see |14p. 

In addition there is evidence that stochastic resonance may be used not 
only at the level of sensory processing, but also in central cognitive processes. 
In a psychophysical study by Usher and Feingold, the memory retrieval of 
arithmetical multiplication was found to be enhanced by the addition of 
noise [T5] . 

In this article, we feed a simplified model of a cortical column with two 
time-varying input signals. We first show that for a broad set of compu- 
tation based on those input signals, addition of a small amount of noise 
enhances the computational power of the system. The location where the 
noise strength is optimal lies approximately where the network reacts the 
strongest to a change in the mean input (maximum susceptibility) . We then 
set the connectivity to zero (with appropriate scaling of the statistics of the 
input to the neurons). Although the simplest task (addition of both input 
signals) can be achieved with a similar accuracy to that obtained with a 
connected network, a multiplicative task can only be solved if the popula- 
tion of neurons is connected. The stochastic resonance effect is thus seen to 
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take place at the system level |16j rather than at the single cell level. It is 
an emergent property of the neuronal assemblies. 

2 Methods 

We consider in our simulations networks of N = 200 cells (leaky integrate- 
and-fire neurons). The connectivity matrix is fixed and every neuron receives 
input from Ce = 40 excitatory and Cj = 10 inhibitory presynaptic neurons 
randomly chosen among the N — 1 neurons in the network. The system is 
made out of 80% excitatory and 20% inhibitory neurons, reflecting the ratio 
of pyramidal cells to interneurons in cortical tissue. This excess of excita- 
tory neurons is approximately balanced by the greater efficacy of synaptic 
transmission for inhibition; in our model six times bigger than for excita- 
tion: loj = 7.2mV and lue = 1.2mV. This approximate balance between 
excitation and inhibition is thought to take place at a functional level in 
cortical areas |17l 118) . Sparsely-connected networks of spiking neurons of 
this type have been fully described in terms of their dynamical behaviour 
|19j . The dynamics of the leaky integrate-and-fire neurons is described by 
the following equation: 

r m ui(t) = - Ui (t) + Rir tw (t) + RIf xt (t) (1) 

where Ui(t) describes the membrane potential of neuron i with respect to its 
equilibrium value Uj = 0, r m = 20ms corresponds to its effective membrane 
time constant and R = 100MJ7 is the effective input resistance of the neuron 
stimulated by a total input RIf etw (t) + RIf xt (t). We add to this equation 
a threshold condition; if the membrane potential of the neuron exceeds the 
critical value 9 = 20mV, a spike is emitted and after a refractory period 
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r rp = 2ms, the integration start again from the reset potential u rese t = OmV. 
Every time the neuron i receives an action potential from a presynaptic 
pyramidal cell j (resp. interneuron) , its membrane potential is depolarized 
according to the value of the synaptic efficacy for excitation (jj = +oje (resp. 
for inhibition, the neuron is hyperpolarized by an amount Uj = — o>j). The 
input a neuron % will get from within the network can thus be written as: 

Rir tW (t) =r m J2 ^ £ S(t - t) - D) (2) 

jeMi k 

where Mj is the ensemble of presynaptic neurons, tj the time neuron j fires 
its fe'th spike and D = 1ms is a short transmission delay. 

We can now decompose the external stimulation RIf xt {t) into two contri- 
butions. First, we model all noise sources, ranging from synaptic bombard- 
ment from neurons outside the network to different sources of noise diversely 
located at the level of synaptic transmission, channel gating, ion concentra- 
tions, membrane conductance to name but a few. All these noise sources 
are grouped into a term JlJ back 9 r with a mean depolarisation RI dep and a 
white noise component defined by its standard deviation RI noise . We can 
define the susceptibility x of the network as the sensitivity of the population 
spiking rate v upon a change in the mean depolarisation RI dep : % = g ®% ep . 
We constrained our simulation space to small mean depolarisation RI dep so 
that in absence of the noisy component RI nmse (t), the network exhibits no 
spiking activity (subthreshold regime). 

[Figure 1 about here.] 

Then, we inject two test signals I\{t) and hit), each to 20% randomly 
chosen neurons in the network. Both inputs share the same statistical prop- 
erties; they are constant over a time interval T = 40ms and then they switch 
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to new randomly chosen values and remain constant for the next time inter- 
val T. At each transition time, the new values are chosen uniformly over the 
interval [— 50pA, 50pA]. We want to know how performance in a series of 
computational tasks based on both test signals Ii(t) and I 2 (f) are affected 
by the noise level. We adapt a paradigm introduced in the framework of Liq- 
uid State Machines [2U] or Echo State Networks [U : we consider a readout 
with a dynamics dy/dt = — {{y — ao)/ r s) + YU=l a i Sfc ~~ ti) where the 
sums run over all firing times t\ of all neurons in the network. t s = 5ms is 
a short synaptic time constant. The N + 1 free parameters a, (0 < % < N) 
are chosen so as to minimize the signal reconstruction error between the 
readout and the target: E = ([y(t) - F{I x {t - A), I 2 (t - A))] 2 ). In our sim- 
ulations we fixed A = 15ms so that the transient period after a transition 
has vanished. The set of functions F(Ii,I 2 ) that are under consideration in 
the present study are; the addition F = Ii + 1 2 , the multiplication F = l\I 2 
which plays a crucial role in the transformation of object locations from 
retinal to body-centered coordinates [2S1 and two polynomials of degree two 
F = (h + h? = l\ + i| + hh and F = (h- I 2 ) 2 = if + if - hh, which 
can be seen as the nonlinear XOR paradigm |22j . 

Parameters were optimized using a first simulation (learning set) lasting 
100 seconds (lOO'OOO time steps of simulation) and were kept fixed after- 
wards. The performance measurements reported in this paper are then 
evaluated on a second simulation of 100 seconds (test set). We compare our 
results to a simple two parameter readout. Such a readout only adjusts to 
the mean of the time series F(Ii(t), ^(t))- The reconstruction error for such 
a readout equals the variance Op of the time series F(Ii(t),I 2 (t)). The per- 
formance with the full readout are therefore expressed as a gain (in percent) 
over the trivial prediction: G = 100(1 — ^-), where E is the error introduced 
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above. 

In the simulation of FigEl an connections in the network were removed. 
In order to compensate the loss of input I netw , we changed the background 
input 

RI backgr go ^ ^ 

neurons receive an input with the same statistical 
properties (mean and variance). The adapted input in the network with "no 
connection" (nc) is thus: Rl^P = RI dep + vT m {CE^E — Ciloj) for the mean 
and RIZ ise = RI noise + ^vT m (C E u? E + C/wf) for the variance term, 1/ is 
the mean population rate in the connected network (see |19j^. 

Simulation results were obtained using the simulation software NEST 1 . 

3 Results 

We want to know how noise influences the ability of a recurrent network of 
spiking neurons to process information and to perform a series of computa- 
tional tasks. In particular we are interested in effects related to stochastic 
resonance. We therefore relate the optimal noise level (where the system has 
a maximum performance) to dynamical properties of the network. From that 
perspective, we analyze how performance in a series of computational tasks 
based on both test signals l\(t) and l2(t) are affected by the noise level. In 
a first series of simulations, we measure the gain over the trivial prediction 
for three different functions of the test signals; F = I\ + I2, F = (Ii + I2) 2 
and F = {I\ — I2) 2 (see figure respectively top right, bottom left and 
bottom right graphs). Note that the function F = (I\ — I2) 2 can be seen 
as an implementation of the XOR task, which cannot be solved by a sin- 
gle layer neural network (perceptron) j^j. In all three tasks, the network 
exhibits stochastic resonance; for a given mean depolarisation RI dep , there 

is a non-monotonic dependence upon the noise level. The maximum gain 
1 NEST Initiative, available at www.nest-initiative.org 
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compared to trivial prediction reaches 38% for the additive task and 7 — 9% 
for polynomials of degree two. A comparison to the map of the susceptibility 
of the network x = gfd ep ( see top left graph of figure HI) indicated that the 
tasks are solved best when the sensitivity of the network upon changes in the 
mean input is highest. Addition of noise both increases the susceptibility of 
the network and the capacity of performing complex computation based on 
sparse inputs. 

[Figure 2 about here.] 

In order to know whether stochastic resonance displayed in the network 
is an emergent property of the population of neuron |16j or a single cell 
effect, we removed all connections within the network. In this second se- 
ries of simulations, we compare the performance achieved in networks with 
connectivity to networks with no connectivity. The latter networks receive 
adapted version of the mean input BI** and of the noisy input RI™° lse so 
that every neurons is stimulated with a mean and variance equivalent to 
that of the connected network (see methods) . 

In the first task; the addition F = I\ + I2, the performance with and 
without connectivity are similar (see figure 01 left). Since the readout unit 
performs a weighted sum that runs over all neurons, it can capture the 
essence of this simple computation by summing the averaged response of 
the groups of neurons receiving input I\ with the averaged response of those 
receiving l<i- Recurrent loops play here no significative role. In the second 
task we train the network to perform the multiplication of the two test sig- 
nals F = J1J2. This arithmetical operation is thought to be essential to 
the brain in order to do coordinate transformation [2Hj. The computation 
of a multiplication by a recurrent neural network was shown to be achiev- 
able in a model of the parietal cortex |241 125j . In this multiplicative task 
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F = I1I2, the complex recurrent network outperforms the network with no 
connectivity (see figure |3] right). In fact, in absence of connections within 
the population of neurons, multiplication cannot be solved by the simple 
addition of noise. Stochastic resonance displayed in the multiplicative task 
therefore takes place at a system level rather than at the level of single 
neurons. 

[Figure 3 about here.] 

4 Conclusion 

Complex networks of neurons fall in the class of non linear systems with a 
threshold; systems that are known to exhibit stochastic resonance. From 
the experimental side, evidences have shown that the phenomenon helps in 
detecting sensory signals of small amplitude, and furthermore to favor high 
level cognitive processes such as arithmetical calculations. Our model has 
revealed the presence of stochastic resonance in a series of neural-based com- 
putation. Whereas simple additive transformations of input signals can be 
solved by a collection of independent neurons, more complex computations 
need the massive recurrence typically observed in cortical tissue. Such com- 
plex tasks include the XOR problem, a nonlinear benchmark test, and the 
arithmetical multiplication the brain is likely to use in order to achieve coor- 
dinate transformation. Stochastic resonance displayed is then an emergent 
property of the brain microcircuitry. 
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Figure Captions 
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1 Structure of the network. All neurons in the network receive in addition to their recurren 
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2 Top left: susceptibility y = j|^r in (Hz/pA) of the network as a function of the statistica 
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3 Left: Gain fin percent) over the trivial prediction for the additive task h + Io: for the coi 



FIGURES 



14 




Figure 1 Structure of the network. All neurons in the network receive in 
addition to their recurrent afferents a background stochastic input made out 
of I dep and I nmse (t). In addition two randomly generated subpopulations, 
each composed of 20% of the total number of neurons, receive test inputs 
I\(t) and h{t). The readout y(t) sees all neurons in the network and is 
trained to match a function of the test input I\ and li (see the methods 
section for details). 
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Figure 2 Top left: susceptibility x = gjde P m (Hz/pA) of the network 
as a function of the statistical properties of the external drive (mean and 
variance). A high susceptibility (light colors) defines a network that is highly 
sensitive to a change in the mean of the drive. Top right and bottom: gain 
(in percent) over the trivial prediction as a function of the mean and variance 
of the external drive; for the simple additive task I\ + I2 (top right), for a 
first polynomial of degree two l\ + if + 2/1/2 (bottom left) and a second 
polynomial of degree two if + if — 2/1/2 (bottom right). Level curves are 
shown for the sake of clarity at 4% and 6% for the polynomials and at 30% 
and 35% for the addition. The system exhibits stochastic resonance for all 
three different tasks. In addition, the location where the performance peak 
approximately corresponds to the zone of high susceptibility. 
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Figure 3 Left: Gain (in percent) over the trivial prediction for the 
additive task I\ + h; for the connected network (solid) and for the control 
network, when the connectivity is set to zero (dashed). I dep = 5.5pA. The 
unconnected collection of neurons is capable of solving this simple task with 
a similar accuracy as the randomly connected neural network. Right: Gain 
(in percent) over the trivial prediction for the multiplicative task I1I2; for the 
reference network (solid) and when the connectivity is set to zero (dashed). 
In absence of recurrence, the network is no longer able to sustain complex 
computations. Stochastic resonance is thus a population-based effect rather 
than a single-cell phenomenon. 



